Information Retrieval from Neurophysiological Signals

One of the ultimate goals of neuroscience is decoding someone’s intentions directly from his/her brain activities. In this thesis, we aim at pursuing this goal in different scenarios. Firstly, we show the possibility of creating a user-centric music/movie recommender system by employing neurophysiological signals. Regarding this, we employed a brain decoding paradigm in order to classify the features extracted from brain signals of participants watching movie/music video clips, into our target classes (two broad music genres and four broad movie genres). Our results provide a preliminary experimental evidence towards user-centric music/movie content retrieval by exploiting brain signals. Secondly, we addressed one of the main issue of the applications of brain decoding algorithms. Generally, the performance of such algorithms suffers from the constraint of having few and noisy samples, which is the case in most of the neuroimaging datasets. In order to overcome this limitation, we employed an adaptation paradigm in order to transfer knowledge from another domain (e.g. large-scale image domain) to the brain domain. We experimentally show that such adaptation procedure leads to improved results. We performed such adaptation pipeline on different tasks (i.e. object recognition and genre classification) using different neuroimaging modalities (i.e. fMRI, EEG, and MEG). Thirdly, we aimed at one of the fundamental goals in brain decoding which is reconstructing the external stimuli using only the brain features. Under this scenario, we show the possibility of regressing the stimuli spectrogram using time-frequency analysis of the brain signals. Finally, we conclude the thesis by summarizing our contributions and discussing the future directions and applications of our research.